Abstract What strategies do sighted individuals utilize to travel in an unfamiliar location and how can blind individuals utilize the same strategies for orientation and mobility travel through the use of assistive technology?
Strategies/Preparation Before traveling to an unfamiliar location you’ll need the following information 1. Address of your Destination: # Street Name, City, State, & ZIP - Yes it really is best to have the zip code 2. Address your leaving from: # Street Name, City, State, & ZIP - Yes it really is best to have the zip code
GPS Apps Then we all plug in the information into our favorite GPS device aka our cell phone. Whether you are searching for directions using a Computer, Android, Windows, or iOS device Maps or Google Maps is the most accessible.
Why Google Maps? Choose your method of travel? - walking - driving - Public transportation
That’s right Google has information about available public transit. It provides links to the schedules and location.
Visuals - Street View: Has a view of the street, stores, intersection types, number for lanes, signs - Terrain View: Overhead view (I never use) - Map: Classic line view
Walking Man Tool: This tool lets you virtually walk the street and follow your route. Low Vision O&M Concepts: - walking around the block - previewing the environment - address systems - choosing landmarks - previewing and intersection
So using Google Maps or iOS Maps both the Sighted and Blind could obtain the route aka list of directions to a given destination.
Visual Exploration of the Map - Sighted Visual Familiarize themselves with the Route: Accessible Views noted above
Apps for Touch Screen GSP/Maps - Blind individuals can also tactually follow the route using iOS Maps.
Other Apps Ariadne GSP - iOS Where the Hell Am I - iOS Intersection Explorer - Android/Google Play Maps - iOS free
Landmarks Well before we go into Landmarks we will have to refine the meaning of this word for the use of assistive technology.
As Orientation and Mobility Specialist a Landmark is defined as something permanent, color attribute, size, shape, and contrast.
Really?
Nowadays landmarks are businesses such as McDonald’s, Burger King, Giant, and even local business names. When previewing my route to a new location click the button to preview local businesses. I think the McDonalds M Signs is the most well know visual landmark. We use the landmarks to gage how we are to our given location and confirm we are on the right path or well at least I do.
Apps for Business and Locations of interest for the Blind: Sendero LookAround - iOS AroundMe - iOS StreetFinder - Android/Google Play MotionX - iOS, Android/Google Play
Building Orientation and Mobility Concepts for Blind Children
While traveling in the car sighted children learn about the area in which they live simply by watching out the window. They learn the name of the street the live on viewing the street signs, their address by looking at the number on the house, and they learn to gauge distance using visual landmarks about how close they are to home once they pass the grocery store.
While traveling in the car with a child who is blind set your GPS to even basic destinations so your child can learn about their environment the same way. It might sound stupid to set you GPS from your house to the grocery store but will teach your child street names, time distance, and directionality. Expand the information spoken to hear surrounding destinations.
Simulate Vision with Technology
So I asked myself if I was lost in an open space (empty parking lot) could I get enough information about my environment to find my way out.
Yes we all know as good O&M Instructors you should listen for the sound of traffic and walk toward it to locate a shore line and find your way out.
But Apps are more Fun! With my iPhone 5 I used the Apps TapTapSee and VizWiz (which both require a 3G/4G or Wi-Fi Connection). Where did I hold my camera to take a picture when I couldn't see it? At eye level as I figured that would work best to capture my visual perspective.
Making a series of quarter turns I took 4 pictures which told me the playground was to my right, the school was behind me, the track was in front of me and the driveway way on the left.
I then took a picture in the direction of the driveway, asked my question to Viz Wiz and got a Good Answer.
How Can These Apps be used in Teaching?
Alana age 7, introduction to TapTapSee Alana travels down the hallway with a long white cane. She confirms her destination by locating the braille sign on the side of the door frame. While traveling down the hallway Alana located the Cafeteria. She then takes a picture of the room using TapTapSee. TapTapSee announces it is a picture of the cafeteria and Alana is amazed. Alana continues to explore the school with her new App finding her way to the office, library, and explored the coat hanger hooks.
Parking Lot The following Day we used the App again to learn about the concept of a parking lot and what’s found there.
What’s a parking lot? Hum... where we play. What else might you find in it? I don't know. Alana uses TapTapSee to take four pictures making quarter turns. TapTapSee states the playground, cars, parking lot. There are cars. Can you hear the cars? No. Can you see the cars? No. That’s because they are parked, their engines are not running and so you cannot hear them. But in a parking lot, people come to park/turn off their cars and leave them. Alana says: Oh so they are just sitting there waiting until they come back and turn them on. Yes Parked. Great can I go to the playground? Sure Alana makes a quarter turn the right remembering the direction she had been facing when she took the picture using TapTapSee.
What's the difference between Tap TapSee and Viz Wiz? Both of these Apps are camera based Apps for the Blind and Visually Impaired and are accessible using the Voiceover Screen Reader.
Basically you take a picture of something, the picture is sent to the Internet, and 30 seconds later the App announces a description of the picture.
Both Apps send the picture to be processed into a database or person which to verify the image and provide a description.
VizWiz lets you ask questions about what you are looking at! You can type your question or even use your voice to record your question. So.....
What Else is VizWiz good for? Hum. Ever wonder why they labels on the vending machine keypads are in Braille when the item number are presented visually behind the glass? It's like labeling the ATM machine at the drive-up bank teller.
Anyway using VizWiz you can take a picture of a Vending Machine and ask what the item number of Coke is.
How Do Those Camera Apps Work?
LookTel Apps like the LookTel Money Reader and the LookTel Recognizer utilize Visual Pattern Recognition Technology.
LookTel sent me this cool App called the MarkerTool. The MarkerTool App I utilize during my presentations demonstrates what kind of information/patterns the camera is looking for aka Visual Patten Recognition. The camera does not look for colors it looks for patterns, shapes, and angles. While holding the camera pointed at an object you will notice several green dots indicating individual points recognized. The points are individual pieces of the overall picture/visual pattern located by the camera. These points are the matching points recognized both individually as well as together using several other points to create the image. This kind of processing ability reminds me of visual closure. For example, while looking at a letter with parts removed your brain will try to fill in the missing pieces to identify the letter. The same process applies to VPR (visual pattern recognition) technology. Even though the picture was taken utilizing several points, if a point is missing the picture/object/landmark is still identified. (See video demonstration link.
(Full Article Link) Read More about Virtual Pattern Recognition - Warning high tech article
Nantworks iD
Nantworks's iD (visual pattern recognition) technology provides a reliable and robust vision solution that truly gives electronic devices the ability to detect and recognize complex visual patterns – in effect, to see.
Thus this technology loves what all of us vision teachers hate "Visual Clutter!" That's right the more items you can get into a picture the greater the probability the picture will be recognized, even if one element is removed or moved.
The combination of several key elements allows the ViPR technology to achieve a high level of performance.
First, is the choice of descriptors it uses to encode unique visual patterns such as the corner of an object or the print on a label. It looks for angles and corners aka Shape. Then it looks for patterns and inner/outer details matching key points to identify an image from a database aka your picture database.
Hallways While standing at the end of the hallway I took a picture looking straight down the hallways. Why would that work? Because the shot was a distance view which allowed the Visual Pattern Recognition Technology to scan a pair more key points to make a match. The Visual Recognition Pattern Recognition looks for corners and angles. It will detect and match changes in building structure such as doorways, overhangs, brick, shelving, etc.
Murals- Always catches the target.
Second, is the ability to analyze a new image to collect sufficient evidence to reliably find a match within an extremely large set of possible candidates? For example, common signs located within in buildings. Since the sign for the bathroom is the same image throughout the building, the VPR technology utilized by your camera will analyze the image, collect matching points, and identify the picture as the women's/men's bathroom regardless of the signs location. Why because the signs are the same and regardless of the background or missing points of the background, the matching points will still identify the sign. Signs Signs can look different or they can be the same both will work.
Examples 1. Common ADA Braille Signs such as Exit, Stairs, Women, and Men. Since these pictures are consistent the LookTel Recognizer will also identify the sign in any location. 2. When dealing with school classroom many teachers can make a creative sign to post on or near their door. This works great for making hallway doors stand out.
Recognition With Recognition at an angle partial occlusion different position at a distance with partial occlusion. This means the regardless of the angle in which you are standing or the distance you are standing from the object, the target/object is still identified.
So let think about that for a second when considering setting a picture landmark using the LookTel Recognizer?
The closer you are to an object the large it is which decreases your field of view. On the flip side the further away you are the greater your field of view aka you can see more at one time.
Therefore when setting a landmarks I've had a lot of success using the LookTel Recognizer for indoor orientation and mobility.
When setting landmarks I have had the most success standing at a distance of 10 - 15 feet way. I create two sets of landmarks one for near at 5 feet and one at a distance 15 - 20 feet away. That way you’re certain to catch a match point no matter where you are standing.
Doors Many of the door ways in building are consistent throughout the building.
Example Set a landmark/take a picture of the doors exiting the building. Since the doors all look the same you will now be notified anytime you are facing an exit.
Third, is the ability to do all this computation in an extremely efficient manner: recognition happens in a fraction of a second when searching a database of several hundred patterns. Basically the VPR technology moves so fast to match points your sign or landmark is processed quickly enough to notify you of the object identified.
Performance/Features 80-100% recognition rate depending on the character of the objects to recognize Works for a wide range of viewing angles, lens distortion, imager noise, and lighting conditions Works even when a large section (up to 90%) of the pattern is occluded from the view by another object • Can simultaneously recognize multiple objects Can handle databases with thousands of visual patterns without a significant increase
QR Codes and Bar Code Scanning Digit-Eyes is a Bar Code and QR Code Scanner. Digit-Eyes like the LookTel Recognizer most likely utilize VPR technology as well. The App requires you to initiate/start the bar code scanner and the camera will automatically search of a bar code. Thus you don’t actually have to take a picture. Utilizing a 3G/4G or Wi-Fi connection Digit Eyes can locate the bar code on an object and identify the object based on the UPC code.
A UPC code is that small white box with vertical lines and numbers on the package with identifies the object based on a database. This works great for identifying food products in the kitchen or grocery store.
A QR Code is that square shaped box with an individual pattern which also identifies an object based on a database.
The Cool thing about Digit-Eyes is their website at www.digit-eyes.com This website allows you to create a free account. The iOS App can be linked to your account by signing using your user name and password. Digit-Eyes will sync your created QR Codes to the App on your phone. Once the database is synced, QR Codes created by you on their website can be scanned an identified without a 3G/4G a Wi-Fi connection.
What kinds of QR Codes can you create using their website?
I create QR Codes to be utilized for indoor orientation and mobility. A Braille sign can on tell you the name of the room and the room number. By creating a QR code and placing it under the Braille sign you can create more detailed tags to identify you location within the building.
For example, a QR Code might contain more detailed information about your current location. Such as, you are facing the cafeteria. Make a quarter turn to the right and travel straight to the Gym. Make a quarter turn to your left and walk straight to locate the office.
In addition, you could create QR codes to identify information about intersections. For example, a QR code can be laminated and posted at an intersection. The information repeated or displayed is specific to what you create within the Digit-Eyes website. Thus you can make your QR Code as specific or vague as you’d like. Such as, You are standing on the North West corner of the intersection on Main and State Street. This is a split-phase traffic light controlled intersection. Each street has 2 parking lanes, 2 driving lanes, and a designated left hand turning lane.
Lastly, Digit-Eyes will allow you to order clothing labels to be sewn into your clothing. Like the QR Codes, Digit-Eyes allow you to create individual labels identifying your clothing using the Digit-Eyes App.
A Picture’s Worth a Thousand Words
Abstract
What strategies do sighted individuals utilize to travel in an unfamiliar location and how can blind individuals utilize the same strategies for orientation and mobility travel through the use of assistive technology?
Strategies/Preparation
Before traveling to an unfamiliar location you’ll need the following information
1. Address of your Destination: # Street Name, City, State, & ZIP
- Yes it really is best to have the zip code
2. Address your leaving from: # Street Name, City, State, & ZIP
- Yes it really is best to have the zip code
GPS Apps
Then we all plug in the information into our favorite GPS device aka our cell phone. Whether you are searching for directions using a Computer, Android, Windows, or iOS device Maps or Google Maps is the most accessible.
Why Google Maps?
Choose your method of travel?
- walking
- driving
- Public transportation
That’s right Google has information about available public transit. It provides links to the schedules and location.
Visuals
- Street View: Has a view of the street, stores, intersection types, number for lanes, signs
- Terrain View: Overhead view (I never use)
- Map: Classic line view
Walking Man Tool:
This tool lets you virtually walk the street and follow your route.
Low Vision O&M Concepts:
- walking around the block
- previewing the environment
- address systems
- choosing landmarks
- previewing and intersection
So using Google Maps or iOS Maps both the Sighted and Blind could obtain the route aka list of directions to a given destination.
Visual Exploration of the Map
- Sighted Visual Familiarize themselves with the Route: Accessible Views noted above
Apps for Touch Screen GSP/Maps
- Blind individuals can also tactually follow the route using iOS Maps.
Other Apps
Ariadne GSP - iOS
Where the Hell Am I - iOS
Intersection Explorer - Android/Google Play
Maps - iOS free
Landmarks
Well before we go into Landmarks we will have to refine the meaning of this word for the use of assistive technology.
As Orientation and Mobility Specialist a Landmark is defined as something permanent, color attribute, size, shape, and contrast.
Really?
Nowadays landmarks are businesses such as McDonald’s, Burger King, Giant, and even local business names. When previewing my route to a new location click the button to preview local businesses. I think the McDonalds M Signs is the most well know visual landmark. We use the landmarks to gage how we are to our given location and confirm we are on the right path or well at least I do.
Apps for Business and Locations of interest for the Blind: Sendero LookAround - iOS AroundMe - iOS StreetFinder - Android/Google Play MotionX - iOS, Android/Google Play
Building Orientation and Mobility Concepts for Blind Children
While traveling in the car sighted children learn about the area in which they live simply by watching out the window. They learn the name of the street the live on viewing the street signs, their address by looking at the number on the house, and they learn to gauge distance using visual landmarks about how close they are to home once they pass the grocery store.
While traveling in the car with a child who is blind set your GPS to even basic destinations so your child can learn about their environment the same way. It might sound stupid to set you GPS from your house to the grocery store but will teach your child street names, time distance, and directionality. Expand the information spoken to hear surrounding destinations.
Simulate Vision with Technology
So I asked myself if I was lost in an open space (empty parking lot) could I get enough information about my environment to find my way out.
Yes we all know as good O&M Instructors you should listen for the sound of traffic and walk toward it to locate a shore line and find your way out.
But Apps are more Fun!
With my iPhone 5 I used the Apps TapTapSee and VizWiz (which both require a 3G/4G or Wi-Fi Connection). Where did I hold my camera to take a picture when I couldn't see it? At eye level as I figured that would work best to capture my visual perspective.
Making a series of quarter turns I took 4 pictures which told me the playground was to my right, the school was behind me, the track was in front of me and the driveway way on the left.
I then took a picture in the direction of the driveway, asked my question to Viz Wiz and got a Good Answer.
How Can These Apps be used in Teaching?
Alana age 7, introduction to TapTapSee
Alana travels down the hallway with a long white cane. She confirms her destination by locating the braille sign on the side of the door frame.
While traveling down the hallway Alana located the Cafeteria. She then takes a picture of the room using TapTapSee. TapTapSee announces it is a picture of the cafeteria and Alana is amazed.
Alana continues to explore the school with her new App finding her way to the office, library, and explored the coat hanger hooks.
Parking Lot
The following Day we used the App again to learn about the concept of a parking lot and what’s found there.
What’s a parking lot? Hum... where we play.
What else might you find in it? I don't know.
Alana uses TapTapSee to take four pictures making quarter turns. TapTapSee states the playground, cars, parking lot. There are cars.
Can you hear the cars? No. Can you see the cars? No. That’s because they are parked, their engines are not running and so you cannot hear them. But in a parking lot, people come to park/turn off their cars and leave them.
Alana says: Oh so they are just sitting there waiting until they come back and turn them on. Yes Parked. Great can I go to the playground?
Sure Alana makes a quarter turn the right remembering the direction she had been facing when she took the picture using TapTapSee.
What's the difference between Tap TapSee and Viz Wiz?
Both of these Apps are camera based Apps for the Blind and Visually Impaired and are accessible using the Voiceover Screen Reader.
Basically you take a picture of something, the picture is sent to the Internet, and 30 seconds later the App announces a description of the picture.
Both Apps send the picture to be processed into a database or person which to verify the image and provide a description.
VizWiz lets you ask questions about what you are looking at!
You can type your question or even use your voice to record your question. So.....
What Else is VizWiz good for?
Hum. Ever wonder why they labels on the vending machine keypads are in Braille when the item number are presented visually behind the glass? It's like labeling the ATM machine at the drive-up bank teller.
Anyway using VizWiz you can take a picture of a Vending Machine and ask what the item number of Coke is.
How Do Those Camera Apps Work?
LookTel Apps like the LookTel Money Reader and the LookTel Recognizer utilize Visual Pattern Recognition Technology.
LookTel sent me this cool App called the MarkerTool. The MarkerTool App I utilize during my presentations demonstrates what kind of information/patterns the camera is looking for aka Visual Patten Recognition. The camera does not look for colors it looks for patterns, shapes, and angles. While holding the camera pointed at an object you will notice several green dots indicating individual points recognized. The points are individual pieces of the overall picture/visual pattern located by the camera. These points are the matching points recognized both individually as well as together using several other points to create the image. This kind of processing ability reminds me of visual closure. For example, while looking at a letter with parts removed your brain will try to fill in the missing pieces to identify the letter. The same process applies to VPR (visual pattern recognition) technology. Even though the picture was taken utilizing several points, if a point is missing the picture/object/landmark is still identified. (See video demonstration link.
(Full Article Link) Read More about Virtual Pattern Recognition - Warning high tech article
Nantworks iD
Nantworks's iD (visual pattern recognition) technology provides a reliable and robust vision solution that truly gives electronic devices the ability to detect and recognize complex visual patterns – in effect, to see.
Thus this technology loves what all of us vision teachers hate "Visual Clutter!" That's right the more items you can get into a picture the greater the probability the picture will be recognized, even if one element is removed or moved.
The combination of several key elements allows the ViPR technology to achieve a high level of performance.
First, is the choice of descriptors it uses to encode unique visual patterns such as the corner of an object or the print on a label. It looks for angles and corners aka Shape. Then it looks for patterns and inner/outer details matching key points to identify an image from a database aka your picture database.
Hallways
While standing at the end of the hallway I took a picture looking straight down the hallways.
Why would that work?
Because the shot was a distance view which allowed the Visual Pattern Recognition Technology to scan a pair more key points to make a match.
The Visual Recognition Pattern Recognition looks for corners and angles. It will detect and match changes in building structure such as doorways, overhangs, brick, shelving, etc.
Murals- Always catches the target.
Second, is the ability to analyze a new image to collect sufficient evidence to reliably find a match within an extremely large set of possible candidates? For example, common signs located within in buildings. Since the sign for the bathroom is the same image throughout the building, the VPR technology utilized by your camera will analyze the image, collect matching points, and identify the picture as the women's/men's bathroom regardless of the signs location. Why because the signs are the same and regardless of the background or missing points of the background, the matching points will still identify the sign.
Signs
Signs can look different or they can be the same both will work.
Examples
1. Common ADA Braille Signs such as Exit, Stairs, Women, and Men. Since these pictures are consistent the LookTel Recognizer will also identify the sign in any location.
2. When dealing with school classroom many teachers can make a creative sign to post on or near their door. This works great for making hallway doors stand out.
Recognition
With Recognition at an angle partial occlusion different position at a distance with partial occlusion. This means the regardless of the angle in which you are standing or the distance you are standing from the object, the target/object is still identified.
So let think about that for a second when considering setting a picture landmark using the LookTel Recognizer?
The closer you are to an object the large it is which decreases your field of view. On the flip side the further away you are the greater your field of view aka you can see more at one time.
Therefore when setting a landmarks I've had a lot of success using the LookTel Recognizer for indoor orientation and mobility.
When setting landmarks I have had the most success standing at a distance of 10 - 15 feet way. I create two sets of landmarks one for near at 5 feet and one at a distance 15 - 20 feet away. That way you’re certain to catch a match point no matter where you are standing.
Doors
Many of the door ways in building are consistent throughout the building.
Example
Set a landmark/take a picture of the doors exiting the building. Since the doors all look the same you will now be notified anytime you are facing an exit.
Third, is the ability to do all this computation in an extremely efficient manner: recognition happens in a fraction of a second when searching a database of several hundred patterns. Basically the VPR technology moves so fast to match points your sign or landmark is processed quickly enough to notify you of the object identified.
Performance/Features
80-100% recognition rate depending on the character of the objects to recognize
Works for a wide range of viewing angles, lens distortion, imager noise, and lighting conditions
Works even when a large section (up to 90%) of the pattern is occluded from the view by another object • Can simultaneously recognize multiple objects
Can handle databases with thousands of visual patterns without a significant increase
QR Codes and Bar Code Scanning
Digit-Eyes is a Bar Code and QR Code Scanner. Digit-Eyes like the LookTel Recognizer most likely utilize VPR technology as well. The App requires you to initiate/start the bar code scanner and the camera will automatically search of a bar code. Thus you don’t actually have to take a picture. Utilizing a 3G/4G or Wi-Fi connection Digit Eyes can locate the bar code on an object and identify the object based on the UPC code.
A UPC code is that small white box with vertical lines and numbers on the package with identifies the object based on a database. This works great for identifying food products in the kitchen or grocery store.
A QR Code is that square shaped box with an individual pattern which also identifies an object based on a database.
The Cool thing about Digit-Eyes is their website at www.digit-eyes.com This website allows you to create a free account. The iOS App can be linked to your account by signing using your user name and password. Digit-Eyes will sync your created QR Codes to the App on your phone. Once the database is synced, QR Codes created by you on their website can be scanned an identified without a 3G/4G a Wi-Fi connection.
What kinds of QR Codes can you create using their website?
I create QR Codes to be utilized for indoor orientation and mobility. A Braille sign can on tell you the name of the room and the room number. By creating a QR code and placing it under the Braille sign you can create more detailed tags to identify you location within the building.
For example, a QR Code might contain more detailed information about your current location. Such as, you are facing the cafeteria. Make a quarter turn to the right and travel straight to the Gym. Make a quarter turn to your left and walk straight to locate the office.
In addition, you could create QR codes to identify information about intersections. For example, a QR code can be laminated and posted at an intersection. The information repeated or displayed is specific to what you create within the Digit-Eyes website. Thus you can make your QR Code as specific or vague as you’d like. Such as, You are standing on the North West corner of the intersection on Main and State Street. This is a split-phase traffic light controlled intersection. Each street has 2 parking lanes, 2 driving lanes, and a designated left hand turning lane.
Lastly, Digit-Eyes will allow you to order clothing labels to be sewn into your clothing. Like the QR Codes, Digit-Eyes allow you to create individual labels identifying your clothing using the Digit-Eyes App.